1. Field of the Invention
The present invention relates to automatic devices that generate an alarm signal when conditions surrounding an unsupervised person indicate the person requires assistance.
2. Background
Remote security monitoring systems in which a video camera is trained on a subject or area of concern and observed by a trained observer are known in the art. Also infant or child monitors that transmit audio to a portable receiver are available in the market. These devices, however, require constant attention in order to provide protection to the subject or area of concern, such as an elderly person, infant, or child.
Machine identification of faces is a technology that is well developed. In GB 2343945A for a system for photographing or recognizing a Face, a controller identifies moving faces in a scene and tracks them to permit image capture sufficient to identify the face or distinctive features thereof. For example, in a jewelry store security system, the system could sound an alarm upon recognizing a pulled-down cap or facemask.
A monitored person""s physical and emotional state may be determined by a computer for medical diagnostic purposes. For example, U.S. Pat. No. 5,617,855, hereby incorporated by reference as if fully set forth herein, describes a system that classifies characteristics of the face and voice along with electroencephalogram and other diagnostic data to help make diagnoses. The device is aimed at the fields of psychiatry and neurology. This and other such devices, however, are not designed for monitoring persons in their normal environments.
EP 0716402B1 describes a method of detecting the number of people entering a train or other space using infrared sensors and fuzzy inference techniques. When the number of people is outside desired limits or unbalanced, the system can generate notices to that effect which may be linked to devices to correct the condition.
UK 2027312A describes a method of detecting the movement of fish using IR cameras generating a standard video signal.
U.S. Pat. No. 4,524,243 describes a system in which a user is required to activate a switch at specified intervals. Failure to do so results in the generation of an inactivity alarm.
U.S. Pat. No. 5,905,436 discloses a system in which the failure of various sensors in a home to be triggered results in the generation of a signal at a central monitoring station indicating such. The disclosure is directed at the supervision of an elderly person living at home.
UK 2179186A describes a system in which, if movement is not detected at a pre-determined time, an alarm is triggered. A warning is given so that the user can reset the switch.
U.S. Pat. No. 6,002,994 discloses a system in which transmitters, placed at strategic locations in a house, are triggered whenever a person is present at the location of a sensor triggering the transmitter. Also, the system employs other inputs attached to devices and appliances that the user is expected to use. The system is trained to recognize normal patterns of use. The transmitters transmit a signal to a central monitor if the normal pattern is not detected.
In this reference, physiological measurements may include the user""s blood pressure, heart rate, body temperature, body weight, and blood glucose level. Non-physiological measurements may include room temperature, ammonia from spilled urine, methane from spoiling food, a presence of smoke, frequency of electrical usage, frequency of water usage, temperature of water flowing from a tap, the user""s movement within the selected environment as indicated by motion sensors, and use of appliances including a toilet, telephone, stove, microwave oven, toaster, oven, refrigerator, freezer, dishwasher, bath, shower, garbage disposal means, clothes washer, clothes drier, mail box, door and vehicle.
In another application area, machines automatically detect an occupant""s presence or specific features of the occupant for purposes of machine-authorization and authentication or convenience. To that end, some prior art systems employ biometric sensing, proximity detectors, radio frequency identification tags, or other devices.
Although automated systems for monitoring the activities of elderly persons have been proposed, there exists a need for enhanced robustness of such systems and an ability to avoid the signaling of false positives and, more importantly, earlier intervention. For example, an elderly person falling out of the sensor range for an extended period could be disabled from a life-threatening problem yet escape detection by a system such as proposed in U.S. Pat. No. 6,002,994. Subtle queues that humans can detect, and that may indicate a problem before it occurs, are not detectable by prior art systems. Also, alarm information may not be sufficiently informative. As a result of these and other drawbacks, machine-monitoring of persons that are alone is not nearly up to the standard afforded by a living attendant.
Briefly, an alarm system monitors conditions of an independent person, yet one requiring some supervision, such as an elderly person living alone at home. The system monitors a variety of independent signals and combines them to recognize subtle cues that may indicate there will be a need for intervention by a supervisor.
As computers are used more widely, and processing power makes possible the use of inexpensive sensors, the ability to control without direct human intervention has become widely discussed. For example, in the home of the future, the heating system, lighting, music systems, appliances, etc. may all migrate toward a regime in which they control themselves rather than requiring the user to control them. In these models, the user""s intentions are inferred by his/her actions. Presence may trigger heat or light activation; television activation may trigger dimming of lights and closing of curtains. In such environments, inexpensive sensor technology is leveraged by computing power to produce useful responses in the system. Future homes may be filled with sensors permitting many features to be implemented by software modifications alone. In such an environment, a rich array of sensor data can be harnessed for many purposes heretofore unrecognized. The present application combines this data in ways that allow useful patterns to be detected in the manner that a human observer can detect subtle patterns. The object application, in this case, is the monitoring of an independent person such as a sick or elderly person.
The invention contemplates an environment employing pervasive sensors, including video and audio, with the application of artificial intelligence technology to permit the recognition of such subtle cues as mood swings, irregular behavior, and presence of unrecognized persons, objects, or animals. When conditions are classified as requiring or suggesting a need for intervention, an informative alarm signal or message containing information about the situation is transmitted to help the recipient understand what is going on. In an embodiment, the alarm signal is a live video and/or audio feed from a camera trained on the person requiring care. In another embodiment, the alarm signal is a symbolic set of data relating to the status and the condition that generated the alarm, for example, the message xe2x80x9cPerson requiring care idle for N hrs,xe2x80x9d xe2x80x9cPresence of unrecognized face,xe2x80x9d xe2x80x9cPhysiognomy indicates distress,xe2x80x9d or xe2x80x9cAbnormally high level of activity.xe2x80x9d In still other embodiments, the system generates responses to stimulate action, such as a response from the monitored person to elicit a recognized behavior or other machine-sensible profile. The alarm signal may be transmitted by phone line, the Internet, or a wireless channel. In still other embodiments, the sensors can adjust themselves, for example, a camera that can zoom on the facial features of an occupant to capture a recognizable image or video sequence.
The field of artificial intelligence and robotics has given rise to technology that enables machines to make sufficient sense of their surroundings to recognize predefined conditions, navigate vehicles, and identify objects and their orientations, for example. Components of systems called autonomous observers have been made in the lab which allow a machine to follow an observer in an area and track escape routes. Other applications of similar technology include video tracking systems that follow a speaker giving a presentation and respond to the speaker""s gesture commands. In embodiments of the present invention, the technology of image and video recognition, audio recognition, and other inputs may be used to infer the condition of a monitored person requiring care.
Artificial Intelligence (AI) principles are used by a classification engine receiving video, audio, and/or other inputs to model a current situation. When conditions are classified as calling for attention (distress event), video, audio, and other data that may be buffered up to the distress event, as well as live data, may be transmitted to a monitor along with an indication of the class to which the recognized event belongs. For example, the audio signal generated by a crying person requiring care may be classified as a xe2x80x9cperson cryingxe2x80x9d condition either alone or in concert with the classification of other data such as video data of the crying person requiring care. Condition classes for a monitor system may include events such as:
1. trigger by a physiological sensor such as a halter heart monitor, breathing sensor, or audio sensor,
2. detection of an unrecognized voice, face, body, or object,
3. abnormal movement of the person such as indicative of an injury, for example, limping,
4. sudden movement consistent with falling, running, normal walking, crawling, etc.,
5. lack of normal movement such as palsy, lack of coordination, too fast as in addressing an emergency, or movement that is abnormally slow,
6. presence of the person requiring care or other individuals in a space and their number,
7. consistency of the clothing, facial features, etc. of the occupants of a space throughout a period of time.
8. loud sounds, normal sounds, and unusual sounds, based upon signature of sound,
9. unusual location of sound sources,
10. occupancy of unauthorized spaces,
11. occupancy patterns, for example whether monitored person is spending unusual amounts of time in a particular space or away from a particular space,
12. patterns suggesting damage to the monitoring system, or objects in the occupied space,
13. voice signatures of unauthorized occupants or unrecognized voice signatures,
14. body habitus and physiognomy of monitored person or other occupants,
15. status of security system in the space,
16. unrecognized objects in occupied spaces or recognized objects being moved or found in unexpected locations,
17. temperature, humidity, sound levels, or other ambient variables out of normal range,
18. presence of an unrecognized face or body pattern,
An event that triggers an alarm condition may be a simple one such as prior art sensors that monitor breathing or crying, or it may be more complex one that integrate multiple inputs into a software (or hardware) network (e.g., neural; Bayesian) that is programmed or configured to identify or recognize the circumstances, or absence of identifiable (familiar) circumstances. Based on this data, the system ultimately classifies the circumstance as an alarm condition or not. Such network devices may include classifiers in the form of neural networks, Bayesian networks, and other techniques for machine-recognition of physical objects and behaviors. The art in this field is rich and rapidly-growing and it is expected that improved means for implementing the current invention will continually become available.
Preferably the classification engine is trainable so that it does not need to rely solely on predefined template patterns for pattern-matching. In an embodiment, however, the raw data is distilled by AI classifiers, such as a video classifier, into a distilled signal permitting explicit rules to be defined in a further stage of classification as to whether an alarm condition exists or not.
The system may be provided with the ability to generate a simulated dialogue to provide for assistance in training, such as asking an occupant to select from among a number of condition classes present in a monitored space at a time when the occupant can observe the monitored space. This may be used by the system to reduce the ambiguity of its combined signals. For example, the simulated dialogue may be generated using xe2x80x9cchatterbotxe2x80x9d technology with a machine-generated persona such as disclosed in the following references, each of which is incorporated in its entirety as if fully set forth herein.
U.S. patent Ser. No. 09/699,606 for Environment-Responsive User interface/Entertainment Device That Simulates Personal Interaction;
U.S. patent Ser. No. 09/686,831 for Virtual Creature Displayed on a Television; and
U.S. patent Ser. No. 09/699,577 for User interface/Entertainment Device That Simulates Personal Interaction and Responds to Occupant""s Mental State and/or Personality.
The persona may, in response to a particular condition (ambiguous classification of extant conditions or just on a random or interval time basis) request information from occupants about present circumstances. The feedback received may be used by the classification engine to further infer the extant conditions and/or relayed to a responsible party along with other information about circumstances. The above applications also discuss the topic of classifying large arrays of inputs to make decisions about occupants.
The invention will be described in connection with certain preferred embodiments, with reference to the following illustrative figures so that it may be more fully understood. With reference to the figures, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in the cause of providing what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.